1. Field of the Invention
The present invention relates to an image processing apparatus and an image processing method. For example, the present invention relates to image processing for detection of an image area having poor color tone of an eye (red-eye).
2. Description of the Related Art
It is well known that flash photography can cause poor color tone of an eye, which is widely known as the red eye effect. The red eye effect is a phenomenon caused by light that is emitted from a flash and incident on an open pupil and that illuminates the retina at the interior back of the eye. The light reflected from the back of the retina brings the red of the capillary blood vessels into an image of a human being or an animal, such as a dog or a cat, captured by using the flash under poorly illuminated conditions. It is likely to cause the red eye effect in the case of persons having the eyes of lighter pigment because the transmittance of the pupil, that is, the crystalline lens is increased as the pigment becomes lighter.
Digital cameras, which have become popular recently, have been increasingly reduced in size, and the optical axes of the lenses tend to be close to the positions of the light sources of flashes in such digital cameras. The red eye effect is generally likely to occur as the positions of the light sources of the flashes become closer to the optical axes of the lenses. This is an important challenge.
In a known method of preventing the red eye effect, an image is captured with the pupil of a subject being closed by emitting a preflash. However, this method undesirably increases the power consumption of a battery, compared with normal image capturing, and the preflash can damage the facial expression of the subject.
Accordingly, many methods of correcting and processing digital image data captured by a digital camera by using a personal computer etc. to reduce the red eye effect have been developed in recent years.
The methods of reducing the red eye effect on digital image data are roughly divided into manual correction, semi-automatic correction, and automatic correction.
In the manual correction, a user uses a mouse, a pointing device including a stylus and a tablet, or a touch panel to specify a red eye area displayed in a display and removes the red eye.
In the semi-automatic correction, a user specifies an area including a red eye to determine a correction range of the red eye from the specified information and removes the red eye. For example, a user specifies an area around both eyes or specifies one point near the eyes with a pointing device. The user determines a correction range from information concerning the specified area or point to remove the red eye.
In the automatic correction, a digital camera automatically detects a correction range of a red eye from digital image data without requiring a special operation by a user, and performs correction of the red eye.
It is necessary for a user to specify a correction point by performing any operation in the manual and semi-automatic corrections. Accordingly, the user is required to perform a complicated operation in which a correction area is specified after enlarging and displaying a neighborhood of an area to be corrected in the image data. Although such an operation is relatively easily performed in, for example, a personal computer system provided with a large display device, it is not easy to perform the operation of enlarging an image and scrolling the enlarged image to specify a correction area in an apparatus, such as a digital camera or a printer, provided with a small area display device.
Various approaches to the automatic correction of the red eye effect, which requires no complicated operation of users and which is effective for apparatuses without larger display devices, have been discussed in recent years.
For example, Japanese Patent Laid-Open No. 11-136498 discloses a method of detecting a flesh-colored area from an image, searching for pixels supposed to include a red eye in the detected area, and correcting the pixels including the red eye. Japanese Patent Laid-Open No. 11-149559 discloses a method of detecting a flesh-colored area, detecting first and second valley areas having lower luminances corresponding to the luminances of the pupils in the detected area, and determining an eye on the basis of the distance between the first and second valley areas. Japanese Patent Laid-Open No. 2000-125320 discloses a method of detecting a flesh-colored area to determine whether the detected flesh-colored area represents a feature of a human being, and detecting a pair of red eye defects in the area and measuring a distance between the red eye defects and a size of the red eye defects to determine the red eye area. Japanese Patent Laid-Open No. 11-284874 discloses a method of automatically detecting whether an image includes a red pupil and, if a red pupil is detected, measuring the position and size of the red pupil to automatically convert red pixels in the image of the pupil into a predetermined color.
However, the proposed methods of automatically correcting the red eye effect have the following problems.
Although the detection of the red eye area on the basis of the detection of the flesh-colored area of a human being or on the basis of a result of the detection of a face by using, for example, a neural network provides higher reliability, it is necessary to refer to a wider area in the image, thus requiring a large memory and a larger amount of calculation. Accordingly, it is difficult to employ such a detection method in a system built in a digital camera or a printer, although the method is suitable for processing in a personal computer including a high-performance CPU operating at a clock rate of several gigahertz and having a memory capacity of several hundred megabytes.
Many methods that have been proposed, in addition to the above examples relating to the automatic correction, use a feature in which the red eye area has a higher saturation than that of a surrounding area to determine the red eye area. However, the determination on the basis of the saturation is not necessarily suitable for persons having the eyes of darker pigment. As widely known, a saturation S is calculated according to Equation (1) where pixel values are given in an RGB (red, green, and blue) system:[Formula 1]S={max(R,G,B)−min(R,G,B)}/max(R,G,B)  (1)where “max(R,G,B)” denotes a maximum value of an RGB component and “min(R,G,B)” denotes a minimum value of the RGB component.
For example, experiments show that the flesh-colored areas of Japanese are concentrated on around 0 to 30 degrees in a hue (0 to 359 degrees). In an HIS (hue, intensity, and saturation) system, a hue angle of around zero represents a red and the hue is approximated to yellow as the hue angle increases. The RGB values have a relationship shown in Expression (2) at the hue angles of 0 to 30 degrees.[Formula 2]R>G>B  (2)
As described above, it is unlikely to cause a bright red eye in the case of persons having the eyes of darker pigment, compared with those having the eyes of lighter pigment.
Japanese have the following estimated pixel values in the red eye area and the flesh-colored area around the eye in view of the above description:
Red Eye Area: (R,G,B)=(109,58,65)
Flesh-colored Area: (R,G,B)=(226,183,128)
In this case, the saturation of the red eye area is equal to “40” and the saturation of the flesh-colored area is equal to “43”, which is approximately the same value as in the red eye area. In other words, it may be impossible to determine the pixels corresponding to the red eye area depending on subjects even in view of the saturation.